Deep Learning of Binary and Gradient Judgements for Semantic Paraphrase

We treat paraphrase identification as an ordering task. We construct a corpus of 250 sets of five sentences, with each set containing a reference sentence and four paraphrase candidates, which are annotated on a scale of 1 to 5 for paraphrase proximity. We partition this corpus into 1000 pairs of sentences in which the first is the reference sentence and the second is a paraphrase candidate. We then train a DNN encoder for sentence pair inputs. It consists of parallel CNNs that feed parallel LSTM RNNs, followed by fully connected NNs, and finally a dense merging layer that produces a single output. We test it for both binary and graded predictions. The latter are generated as a by-product of training the former (the binary classifier). It reaches 70% accuracy on the binary classification task. It achieves a Pearson correlation of .59-.61 with the annotated gold standard for the gradient ranking candidate sets.